Implementation of machine vision at a mobile device can support a variety of applications enables simultaneous localization and mapping (SLAM), which in turn can support a variety of applications, such as augmented reality (AR) applications and virtual reality (VR) applications, by enabling identification of objects within the local environment of a device through the analysis of imagery of the local environment captured by the device and through correlating pose information for the mobile device with the identified objects. To support these features, the mobile device can collect a wide variety of data, including by capturing imagery at one or more image sensors and by receiving data from a variety of non-visual sensors (e.g., accelerometers and gyroscopes). However, the density and size of the collected information can slow down the identification of objects as well as negatively impact the storage resources of the mobile device.